A 3D Feature-Based Tracker for Tracking Multiple Moving Objects with a Controlled Binocular Head

Object tracking is an important task for active vision and robotics. This paper presents a 3D feature-based tracker for tracking multiple moving objects with a computer-controlled binocular head. Our tracker operates in two phases: an initialization phase and a tracking phase. In the initial-ization phase, correspondence between 2D features in the first stereo image pair is determined reliably by using the epipolar line constraint and the mutually-supported consistency. In the tracking phase, the feedback loop is established by first predicting new 3D feature locations with the Kalman filters and then projecting them onto the 2D images to guide the extraction of 2D features in the new image pair. Here, we propose a RANSAC-based clustering method for motion segmentation and estimation by using the principle of rigid body consensus which states that all the extracted 3D features on a rigid body have the same 3D motion. This new method leads to a feature-clustering algorithm which provides a systematic way for managing splitting, merging, appearance and disappearance of multiple moving rigid objects –– including articulated objects, such as robot manipulators. By using the motion estimates obtained with the RANSAC-based method as the measurements for the Kalman filters, we are able to use linear Kalman filters for predictive visual tracking, instead of the extended Kalman filters which most people used for tracking. Experiments have shown that our tracking system does give good results and can serve as a robust 3D feature tracker for an active binocular vision system.-2